It is frequently useful to construct a 3D computer model of a real world scene. This may be the case, for example, in an augmented reality (AR) device in which a composite image, with mixed real and non-real objects, is used to present a scene to a viewer using the AR device. In certain situations when constructing a 3D model of a real world scene, physical objects or surfaces may be modeled ambiguously or inaccurately and may not be identifiable or positioned correctly in the 3D model. Also, certain objects or surfaces may not be clearly distinguishable in the 3D model from other objects or surfaces that are nearby. In situations in which the 3D model includes these types of ambiguities or errors it may be desirable to configure a device, such as an AR device, to identify or locate points, objects or surfaces as accurately as possible in the real world scene despite possible ambiguities and inaccuracies in the 3D model.